Estimating the Determinants of Health Literacy for Policy Prioritisation

Nathan Green

Department of Statistical Science, UCL

Outline

  • Background
  • Problems
  • Solutions
    • Multilevel regression and post-stratification (MRP)
    • Predictive comparisons
    • Prioritisation with SUCRA
  • Main results
  • Sensitivity analysis
  • Conclusion

Resources

Slides and code here: github.com/n8thangreen/data-science-in-health-talk

Background

  • Health literacy is broadly defined as the ability to access, understand, appraise, and communicate health information, enabling individuals to engage in healthcare and maintain good health throughout their lives.

  • UCL Public Policy Fellowship

  • Focusses on Newham, a diverse borough in East London that faces unique challenges
  • Identified as having some of the lowest levels of health literacy in the UK by University of Southampton (https://healthliteracy.geodata.uk/)

Previous method: Synthetic estimation

  • Weighted Logistic Regression with Synthetic Estimation (Laursen et al. (2016))
    • Frequentist single-level regression with poststratification
  • Used in geography (Gonzalez (1973); Rao and Molina (2015))
  • Can be viewed as the simpler predecessor to MRP
    • Ignores any unique local factors
    • MRP includes shrinkage via random effects
  • A linear model can be thought of equivalent to Regression-Synthetic Estimator at Unit Level
    • Like Simulated Treatment Comparison in HTA
  • Residual-adjusted synthetic estimation similar to Targeted Maximum Likelihood Estimation (TMLE) in causal inference

Problem

  • What are the ‘drivers’ of health literacy, specific to Newham?
  • Can we quantify them?
  • What would happen to health literacy if we were to intervene to effect one of these?
  • Pass kernel / root of distribution name

Data

  • Newham Residents Survey 2023 (NRS)

  • Skills for Life Survey 2011

  • Additional data

    • Labour Force Survey (LFS)
    • UK Programme for the International Assessment of Adult Competencies (PIAAC) 2023
    • Skills for Life Survey 2003

Mutlilevel regression and post-stratification

The predicted probability \(\hat{\pi}_i\) is defined as: \[ \hat{\pi}_i = \text{logit}^{-1} \left( \hat{\beta}_0 + \sum_{x} \hat{\beta}^{x}_{\gamma_x[i]} \right) \]

where \(\hat{\beta}_0\) is the intercept, \(\hat{\beta}^{x}_{\gamma_x[i]}\) are coefficients for covariates \(x\) (age, sex, eng, white, ukborn, qual, inc, job, work, home), and \(\gamma_x[i]\) represents the level or category for covariate \(x\) for individual \(i\). IMD is included as multilevel random effects \(\beta^{\text{IMD}}_j \sim \text{N}(\mu_{\text{IMD}}, \sigma_{\text{IMD}}^2)\). Priors distributions for fixed effects are normal distributions centered at zero with modest variance, and half-normal priors are used for random effect standard deviations .

The health literacy probabilities for each demographic category (cell \(c\)) are weighted by their proportion in the actual Newham population. With 11 covariates resulting in \(|\mathcal{S}|\) = 13,824 cells, the post-stratified estimate \(\hat{\pi}^{\text{mrp}}\) is: \[ \hat{\pi}^{\text{mrp}} = \sum_{c = 1}^{|\mathcal{S}|} w_c \hat{\pi}_{c} \] where \(\mathcal{S}\) is the set of all covariate combinations, \(N_c\) is the population frequency for cell \(c\), \(N\) is the total population size, and \(w_c = N_{c} / N\) are the combination weights.

Predictive comparisons 🤔

  • Terminology borrow from (Gelman?)
  • Like average treatment effects without the causal interpretation

Raking

Priority ranking

  • SUCRA taken from network meta-analysis (NMA)

Main results 👍

  • Performance: Generated code can be highly optimized for a specific task, as it doesn’t need to carry the overhead of conditional logic for other scenarios.

Sensitivity analyses 👍

  • Easier Maintenance: You maintain one core piece of code. Changes to the underlying logic only need to be applied in one place.

Other data sets

Reference

Green, N., Kurt, M., Moshyk, A., Larkin, J. and Baio, G. (2025), A Bayesian Hierarchical Mixture Cure Modelling Framework to Utilize Multiple Survival Datasets for Long-Term Survivorship Estimates: A Case Study From Previously Untreated Metastatic Melanoma. Statistics in Medicine, 44: e70132. https://doi.org/10.1002/sim.70132

Conclusions

Thanks 🙏

References

Gonzalez, Maria E. 1973. “Use and Evaluation of Synthetic Estimates.” In Proceedings of the Social Statistics Section, American Statistical Association, 33–42. American Statistical Association.
Laursen, Kamilla R., Paul T. Seed, Joanne Protheroe, Michael S. Wolf, and Gill P. Rowlands. 2016. “Developing a Method to Derive Indicative Health Literacy from Routine Socio-Demographic Data.” Journal of Health Care Communications 1 (4): 1–9. https://doi.org/10.4172/2472-1654.100033.
Rao, J. N. K., and Isabel Molina. 2015. Small Area Estimation. 2nd ed. Wiley Series in Survey Methodology. John Wiley & Sons.